The past two decades have seen first the displacement of traditional film-based photography by digital photography and then the displacement, still going on at the time of this writing, of digital compact cameras by smartphones and similar multimedia devices with built-in cameras. This gives the consumer the opportunity always to have a camera with him, ready to take a picture when the opportunity arises. Yet this development has also brought a certain degradation in quality. Despite the impressive advances in miniature cameras small and cheap enough to be integrated into a smartphone, the laws of physics set limits to the noise and dynamic range of a tiny sensor behind a tiny lens mounted in a device that gets warm during operation. The quality of the images shot with these smartphones is thus often suboptimal, at least when compared with higher-grade dedicated cameras.
Yet the smartphone revolution has also brought us new opportunities in digital imaging. Smartphones today come equipped with impressively powerful central processing units (CPU) and graphics processing units (GPU) that can perform a lot of mathematical operations in a short time.
It is, therefore, desirable to restore the image quality lost by sensor miniaturization and cost reduction in smartphone cameras by novel uses of the mathematical processing power now available.
A quick review of the patent literature pertinent to the topic:
U.S. Pat. No. 7,518,615 mentions the possibility of using a floating-point frame buffer in order to display an “image recorded by a video camera in very low light, or an infrared image.”
U.S. Pat. No. 8,643,769 contemplates a system for specialized applications where image data are captured and processed as floating-point numbers, and U.S. Pat. No. 6,538,593 teaches in a similar direction.
European patent application EP 0,489,552 teaches the use of a processor that can simultaneously execute a non-floating point operation and one or a few floating-point operations in an imaging system that first converts image data from high-dynamic range integer representation to floating point representation and then back to integer representation at a lower dynamic range. A similar system taught is in U.S. Pat. No. 7,522,756.
U.S. Pat. Nos. 8,503,539 and 8,634,476 teach the use of a GPU “designed to perform fast execution of integer and floating-point arithmetic” in an image-capture apparatus.
U.S. Pat. No. 8,452,090 and application WO 2014/080068 discuss the use of GPUs for Bayer demosaicking. U.S. Pat. No. 8,452,090 appears to be implying that this processing be done representing luminance data as 8-bit integer numbers since it suggests the use of the RGBA storage format, even though a camera image does not have an alpha channel, which with present GPU architectures appears to make sense primarily for an encoding in 8-bit integers for each channel since the addition of an unused alpha channel in that case makes each pixel 32 bits wide, which brings advantages for memory access.
US Patent Applications 2015/0054999 and 20157/00575001 teach a camera image sensor that out-puts luminance readings as floating-point numbers.